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What  is  Data  Replication  ? 


Systematic  propagation  and  maintenance  of 
copies  of  data  between  datastores  within  a 
distributed  environment 
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r*i>  Why  Replicate  Data  in  the  Tactical  Domain  ? 


‘Network-Centric  Warfare’ 

-  ‘An  information-superiority  enabled  concept  of 
operations  that  generates  increased  combat  power  by 
networking  sensors,  decision  makers  and  shooters  to 
achieve: 


shared  awareness 
increased  speed  of  command 
higher  tempo  of  operations 
greater  lethality 
increased  survivability 
a  degree  of  self- synchronization 


D.S.  Alberts,  J.J.  Garstka,  F.P.  Stein,  ‘Network  centric  warfare:  developing 
and  leveraging  information  superiority’,  CCRP  Publication  Series,  1999 
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Tactical  Communications  Constraints 


•  Units  are  highly  mobile 

•  Communicate  by  radio  (voice  and  data;  voice  only; 
data  only) 

-  shared  (broadcast)  medium 


-  connectionless 


Low  data  throughput  ( <  1  kbit/second  for  CNR(P)) 

Variable  data  throughput 

-  highly  dependent  upon  traffic  load  on 
communications  network 


Unreliable  links  (frequent  disconnection,  high  bit 
error  rates) 
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Data  Distribution  Requirements 
in  Tactical  Wireless  Domain 


Autonomous  cooperating  nodes 

-  disconnected  operation 

Peer-to-peer  model  (not  client-server) 

-  avoid  single  point  of  failure 

Propagate  updates  asynchronously  on  ‘all-informed’  basis 

-  profit  from  (shared)  broadcast  medium 

-  change  role  without  substantial  one-time  data  transfer 

-  recover  data  from  any  node 

Data  recovery  must  be  carefully  managed  (bandwidth  issue) 

Data  ownership  an  important  issue  (integrity  and  bandwidth) 

-  single  ownership  of  data  to  avoid/minimize  data  conflicts 

Negative  acknowledgement  scheme 
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Desirable  Characteristics  of  Replication 
for  Tactical  Wireless  Domain 


•  Network  Topology  -  peer-to-peer 

-  avoid  single  point  of  failure 
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Synchronous  vs  Asynchronous  Replication 


•  Synchronous  replication  provides  ‘tight  consistency’ 
between  data  stores  through  two-phase  commit 
protocol  (update  to  originating  and  replicate  database 
occurs  at  same  time) 

-  requires  high  network  availability  and  bandwidth 


-  not  practical  in  tactical  wireless  domain 

Asynchronous  replication  provides  ‘loose 
consistency’  between  data  stores. 

-  There  is  latency  before  data  consistency  is  achieved  because 
replication  occurs  some  time  after  originating  transaction. 

-  used  in  tactical  wireless  domain 
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Desirable  Characteristics  of  Replication 
for  Tactical  Wireless  Domain 


•  Asynchronous  Replication 

-  most  commercial  replication  mechanisms  support  tight 
consistency,  or  loose  consistency  but  assume  latency  not  an  issue 

-  in  reality,  ‘loose  consistency’  often  not  achievable  in  tactical 
wireless  domain 


-  due  to  low  &  variable  throughput  of  tactical  comms  network, 
some  replicated  data  may  not  reach  its  destination 

-  must  live  with  a  state  of  ‘lazy  consistency’,  in  which  the 
datastores  never  fully  synchronize  and  it  is  always  the  case  that,  at 
any  given  time,  some  data  values  will  be  inconsistent. 

-  mechanism  should  protect  consistency  of  high-value  information 
when  network  performance  degrades  (graceful  degradation) 


Propagation  of  database  changes 

-  bandwidth-efficient  when  replicate  only  values  that  have  changed 
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Asynchronous  Replication  - 
communication  types 


Database-to-database 


-  collect  process  :  select  data  to  share  from  primary  data  source 

-  distribution  process  :  deliver  replication  package  to  targets 

-  apply  process  :  apply  database  changes  at  target 

-  implemented  as  middleware,  high  application  transparency, 
preserves  transactional  integrity 


Process-to-process  (publish/subscribe  messaging) 

-  publish  process  (part  of  originating  event) 

-  distribution  process  (usually  store-and- forward) 

-  subscribe  process  (interested  processes  receive  message) 

-  apply  process  (in  accord  with  pre-defmed  business  rules) 

implemented  as  middleware  with  simple  APIs  (not  application 
transparent),  preserves  transactional  integrity 
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->y7  Data  Ownership  Models  for 
Asynchronous  Replication 


•  Master/Slave 

•  each  individual  data  element  has  only  one  ‘owner’  (primary 
source)  with  the  right  to  modify  the  data  value 

•  modified  value  replicated  to  target  replicas,  and  applied 

•  facilitates  data  reconciliation  and  recovery 

•  Update-Anywhere  (aka  peer-to-peer  or  symmetric  replication) 

•  no  designated  master  (primary  source)  for  data  element 

•  any  participating  data  store  can  modify  the  element’s  value  and 
replicate  the  change  to  other  data  stores,  where  it  is  applied 

•  difficult  to  track  author  of  data  change;  enables  data  conflicts 

•  Hybrid  (Two-Way  Master-Slave) 

•  each  participating  data  store  acts  as  ‘owner’  (primary  source) 
for  a  subset  of  total  data  set;  is  primary  source  (sender)  for 
certain  data  changes  and  target  (recipient)  for  other  changes 
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Desirable  Characteristics  of  Replication  for 
Tactical  Wireless  Domain  -  Summary 


Peer-to-peer  network  model 
Broadcast  protocol  (connectionless) 
Asynchronous  replication 

-  enforcing  ‘lazy  consistency’ 

•  graceful  degradation  protecting  consistency 
of  high-value  data 

-  database-to-database  communication 

-  propagate  only  what  has  changed 


-  two-way  master-slave  data  ownership  model 
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Army  Tactical  Command  Control  and 
Information  System  (ATCCIS)  Background 


ATCCIS  Objectives 

-  16  NATO  nations 

-  Interoperability  between  C2ISs 

-  Software/Hardware/ Vendor-independent  Solution 

-  Two  Main  Products:  Common  Data  Model  and  ARM 
(ATCCIS  Replication  Mechanism)  specification 

-  Database-to-Database  Replication 

MIP  (Multilateral  Interoperability  Programme) 

-  Goal:  To  Field  an  Interoperability  Solution 

-  Adopted  ATCCIS  Products 

-  Merged  with  ATCCIS  in  2002 
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ATCCIS  and  OSI  Layers 


OSI-Layers 


C2  Applications 
C2  Databases 


7  -  Application 

ATCCIS 

Replication 

Mechanism 

Data  Manager 

ReplicationManager 

'“HIM 

6  -  Presentation 

Transfer  Facility 
Manager 

5  -  Session 

Transfer  Facility 

4  -  Transport 

3  -  Network 

2  -  Data  Link 

1  -  Physical 

ATCCIS  Concept  of  Operation 


National  C2  System 
Processing  and  Presentation 


* 


National  Database 


Conceptual  Data  Model 


ATCCIS  Replication  Mechanism 


Data  Transfer  Protocols 


Physical  Data  Storage 

Standard  Data  Definitions 

Common  Replication 
Architecture 

Communications  Link 


[  \tj  ATCCIS  Replication 


•  Replication  Contracts 

-  “Negotiated  Push” 

-  Agreement  by  both  Parties  ->  Automated  Exchange 

•  Filters 

-  On  Data  Value  and  Data  Source,  Simple  and  Complex 


Payload  Reduction 

-  Reference  Data  +  Transmission  Efficiency  Rules 

Replication  Messages 

-  Incremental  Update  (new/changed  data  only) 

-  Bulk  Update  (for  synchronization) 

-  Control  Messages  (e.g.  activate  node,  propose  contract) 
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Desirable  Characteristics  of  Replication  for 
Tactical  Wireless  Domain  -  Summary 


ARM 

YES 

NO 

YES 

NO 

NO 

YES 

YES 


-  two-way  master-slave  data  ownership  model  YES 
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Peer-to-peer  network  model 
Broadcast  protocol  (connectionless) 
Asynchronous  replication 

-  enforcing  ‘lazy  consistency’ 

•  graceful  degradation 

-  database-to-database  communication 

-  propagate  only  what  has  changed 


What  is  Missing  ? 


•  In  the  tactical  wireless  domain,  for  optimum 

performance  the  replication  protocol  must  be  able  to 
sense,  and  adapt  its  behaviour  to,  the  constantly 
varying  state  of  the  communications  network 


A  Replication  Transport  Layer  must  be  installed  that 
sits  between  the  Replication  Mechanism  and  the 
network  layers 

The  Replication  Transport  Layer  should 

-  take  advantage  of  shared  medium 

-  use  a  standard  connectionless  transport  layer  (UDP  not  TCP) 

-  sense  and  adapt  to  varying  state  of  comms  network 
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ry*,V  Functional  Requirements  for  Replication 
^  Mechanism  and  Replication  Transport  Layer 


Replication  Mechanism  (RM)  must 

-  determine  when  replication  is  to  occur  (context-sensitive) 

-  determine  what  is  to  be  replicated 

-  assemble  the  replication  Protocol  Data  Unit  (PDU) 

-  apply  received  PDUs 

Replication  Transport  Layer  (RTL)  must  support 

-  prioritization  at  PDU  level  (sensitive  to  time-varying  network  state) 

-  retransmission  protocol  (sensitive  to  time-varying  network  state) 

-  degree  of  fault  tolerance 

-  fragmentation/ defragmentation  of  PDU s 

-  acknowledgement  scheme  (negative  ACK) 


Defence  R&D  Canada  -  Valcartier  •  R  &  D  pour  la  defense  Canada  -  Valcartier 

UNCLASSIFIED  -  APPROVED  FOR  PUBLIC  RELEASE 


Functional  Requirements  Delivered  by 
Combination  of  RM  and  RTL 


•  Determine  level  of  effort  allocated  to  PDU  Tx 


-  based  on  importance  of  PDU  content 

•  number  of  retransmission  attempts 

•  choice  of  class  of  transport  service  (guaranteed,  best  effort) 


Track  and  enforce  data  ownership 

-  authority  structure  for  management  of  database  keys 

Mediate  dependencies  on  other  system  components 

-  dependency  of  RM  on  characteristics  of  a  particular  DBMS 

-  dependency  of  RTL  on  characteristics  of  particular  DM  or  RM 
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Conclusions 


•  In  the  tactical  domain,  data  replication  is  key  to 
shared  situational  awareness 

•  To  be  effective  in  this  domain,  replication 
mechanism  and  protocols  must  be  capable  of  sensing 
and  adapting  to  the  changing  state  of  the 
communications  network 


Commercial  replication  mechanisms  support  ‘tight 
consistency’,  or  ‘loose  consistency’  but  assume 
latency  (time  to  resynchronize)  not  an  issue 

No  replication  mechanism  developed  to  date  has 
proven  fully  effective  in  the  tactical  military  domain 
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